Learned perception systems for self-driving vehicles

2022 Spring.Includes bibliographical references.Building self-driving vehicles is one of the most impactful technological challenges of modern artificial intelligence. Self-driving vehicles are widely anticipated to revolutionize the way people and freight move. In this dissertation, we present a collection of work that aims to improve the capability of the perception module, an essential module for safe and reliable autonomous driving. Specifically, it focuses on two perception topics: 1) Geo-localization (mapping) of spatially-compact static objects, and 2) Multi-target object detection and tracking of moving objects in the scene. Accurately estimating the position of static objects, such as traffic lights, from the moving camera of a self-driving car is a challenging problem. In this dissertation, we present a system that improves the localization of static objects by jointly optimizing the components of the system via learning. Our system is comprised of networks that perform: 1) 5DoF object pose estimation from a single image, 2) association of objects between pairs of frames, and 3) multi-object tracking to produce the final geo-localization of the static objects within the scene. We evaluate our approach using a publicly available data set, focusing on traffic lights due to data availability. For each component, we compare against contemporary alternatives and show significantly improved performance. We also show that the end-to-end system performance is further improved via joint training of the constituent models. Next, we propose an efficient joint detection and tracking model named DEFT, or "Detection Embeddings for Tracking." The proposed approach relies on an appearance-based object matching network jointly learned with an underlying object detection network. An LSTM is also added to capture motion constraints. DEFT has comparable accuracy and speed to the top methods on 2D online tracking leaderboards while having significant advantages in robustness when applied to more challenging tracking data. DEFT raises the bar on the nuScenes monocular 3D tracking challenge, more than doubling the performance of the previous top method (3.8x on AMOTA, 2.1x on MOTAR). We analyze the difference in performance between DEFT and the next best-published method on nuScenes and find that DEFT is more robust to occlusions and large inter-frame displacements, making it a superior choice for many use-cases. Third, we present an end-to-end model to solve the tasks of detection, tracking, and sequence modeling from raw sensor data, called Attention-based DEFT. Attention-based DEFT extends the original DEFT by adding an attentional encoder module that uses attention to compute tracklet embedding that 1) jointly reasons about the tracklet dependencies and interaction with other objects present in the scene and 2) captures the context and temporal information of the tracklet's past observations. The experimental results show that Attention-based DEFT performs favorably against or comparable to state-of-the-art trackers. Reasoning about the interactions between the actors in the scene allows Attention-based DEFT to boost the model tracking performance in heavily crowded and complex interactive scenes. We validate the sequence modeling effectiveness of the proposed approach by showing its superiority for velocity estimation task over other baseline methods on both simple and complex scenes. The experiments demonstrate the effectiveness of Attention-based DEFT for capturing spatio-temporal interaction of the crowd for velocity estimation task, which helps it to be more robust to handle complexities in densely crowded scenes. The experimental results show that all the joint models in this dissertation perform better than solving each problem independently

End-to-end learning framework for circular RNA classification from other long non-coding RNAs using multi-modal deep learning.

Over the past two decades, a circular form of RNA (circular RNA) produced from splicing mechanism has become the focus of scientific studies due to its major role as a microRNA (miR) ac tivity modulator and its association with various diseases including cancer. Therefore, the detection of circular RNAs is a vital operation for continued comprehension of their biogenesis and purpose. Prediction of circular RNA can be achieved by first distinguishing non-coding RNAs from protein coding gene transcripts, separating short and long non-coding RNAs (lncRNAs), and finally pre dicting circular RNAs from other lncRNAs. However, available tools to distinguish circular RNAs from other lncRNAs have only reached 80% accuracy due to the difficulty of classifying circular RNAs from other lncRNAs. Therefore, the availability of a faster, more accurate machine learning method for the identification of circular RNAs, which will take into account the specific features of circular RNA, is essential in the development of systematic annotation. Here we present an End to-End multimodal deep learning framework, our tool, to classify circular RNA from other lncRNA. It fuses a RCM descriptor, an ACNN-BLSTM sequence descriptor, and a conservation descriptor into high level abstraction descriptors, where the shared representations across different modalities are integrated. The experiments show that our tool is not only faster compared to existing tools but also eclipses other tools by an over 12% increase in accuracy. Another interesting result found from analysis of a ACNN-BLSTM sequence descriptor is that circular RNA sequences share the characteristics of the coding sequence

GUIDED WAVE PROPAGATION IN THERMAL MEDIA THROUGH THE SEMI ANALYTICAL FINITE ELEMENT METHOD

In this paper, the issue of the estimation of wave propagation characteristics in thermal media is dealt with. A formulation, named the Thermal Semi Analytical Finite Element, based on the semi analytical finite element approach coupled with the thermal effect is offered. Temperature variations affect the mechanical properties of the waveguide. The question of dispersion curves and group velocities is studied. This study is expected to be of use in the sensitivity analysis of guided waves for wave propagation in thermal environment. Comparisons between numerical and analytical results are given to show the effectiveness of the proposed approach

Ugađanje otpora rotora vektorski upravljanog indukcijskog motora korištenjem TS neizrazite logike

In this paper, we focus on the estimation of the rotor resistance to online tune the controllers in case of the Indirect Rotor Field Orientation Control (IRFOC) of Induction Machine (IM). The proposed method is based on the development of an adaptive Takagi-Sugeno (TS) fuzzy flux observer, described in a d-q synchronous rotating frame, to concurrently estimate the IM states and the rotor resistance variation. An investigation of the local pole placement is carried out in order to guarantee both the stability and specified observer dynamic performances. The observer\u27s gains design is based on the resolution of sufficient conditions driven into LMIs terms (Linear Matrix Inequalities). Simulation and experimentation are carried out to show the effectiveness of the proposed results.U ovom radu fokusiramo se na estimaciju otpora rotora za ugađanje parametera kontrolera tijekom rada indukcijskog motora (IM) upravljanog metodom indirektne kontrole orijentacije polja rotora (IRFOC). Predložena metoda je bazirana na razvoju adaptivnog Takagi-Sugeno (TS) neizrazitog obzervera toka, opisanog u d-q sinkronom rotacijskom okviru, kako bi se istovremeno estimirala stanja i varijacije otpora rotora IM-a. Provedeno je istraživanje lokalnog postavljanja polova kako bi se osigurala stabilnost i zadane dinamičke performanse obzervera. Dizajn pojačanja estimatora baziran je na rješenju dovoljnog broja uvjeta izraženih pomoću LMN izraza (linearne matrične nejednakosti). Simulacija i eksperimenti su provedeni kako bi se pokazala ispravnost predloženih rezultata

Looking Ahead: Anticipating Pedestrians Crossing with Future Frames Prediction

In this paper, we present an end-to-end future-prediction model that focuses on pedestrian safety. Specifically, our model uses previous video frames, recorded from the perspective of the vehicle, to predict if a pedestrian will cross in front of the vehicle. The long term goal of this work is to design a fully autonomous system that acts and reacts as a defensive human driver would --- predicting future events and reacting to mitigate risk. We focus on pedestrian-vehicle interactions because of the high risk of harm to the pedestrian if their actions are miss-predicted. Our end-to-end model consists of two stages: the first stage is an encoder/decoder network that learns to predict future video frames. The second stage is a deep spatio-temporal network that utilizes the predicted frames of the first stage to predict the pedestrian's future action. Our system achieves state-of-the-art accuracy on pedestrian behavior prediction and future frames prediction on the Joint Attention for Autonomous Driving (JAAD) dataset

De-anonymizing BitTorrent Users on Tor

Some BitTorrent users are running BitTorrent on top of Tor to preserve their privacy. In this extended abstract, we discuss three different attacks to reveal the IP address of BitTorrent users on top of Tor. In addition, we exploit the multiplexing of streams from different applications into the same circuit to link non-BitTorrent applications to revealed IP addresses.Comment: Poster accepted at the 7th USENIX Symposium on Network Design and Implementation (NSDI '10), San Jose, CA : United States (2010